Validity Issues in the Use of Social Network Analysis for the Study of Online Communities

There is a natural match between studies of online communities and social network analysis (SNA). Information Systems research, in particular, has drawn heavily on the growing data sources available as a by-product of increased online social interaction, conducting social network analyses with this “trace data”. However, this type of data has properties that are distinct from the data upon which researchers in Sociology and Anthropology have typically developed social network measures and their interpretations. This paper examines validity issues deriving from the use of trace data in IS studies of online communities, arguing that more attention needs to be paid to the chain of logic linking data collection, construct development, operationalization and measure interpretation. Four issues are elaborated and illustrated from the literature: 1) unexpected reliability concerns, 2) the need to argue and test the link from ‘found data’ to theoretical constructs, 3) validity problems arising from the stability of the constructs over time and 4) counter-intuitive validity issues deriving from the data being near-complete records of communities’ activities. The paper concludes with recommendations for researchers and reviewers designed to increase the rigor and relevance of SNA research in Information Systems.